Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for custom clipboard formats #6223

Open
wants to merge 29 commits into
base: master
Choose a base branch
from

Conversation

minetoblend
Copy link

@minetoblend minetoblend commented Mar 24, 2024

Allows copying values with custom formats to the clipboard. For now only writes custom formats to the system clipboard on windows, other platforms will store the value in-memory for now. I only did windows for now to not blow up the PR in size, but I plan to contribute support for the remaining platforms as well in the future (Afaik currently only supported on MacOS besides windows, linux would require an upgrade to SDL3).
For simplicity, I limited the custom formats to be text-only.

Adds support for web custom formats on windows, for interoperability with the browser clipboard API.

Web custom formats expects a clipboard entry containing a map of all custom formats in json. The name of the clipboard entries depends on the platform. For windows these entries are expected to be in the Web Custom Format Map entry:

{
  "text/custom" : "Web Custom Format0",
  "image/custom" : "Web Custom Format1"
}

Considerations

The SetData method returns a boolean since SetImage needs to return a boolean currently. I wasn't entirely sure how to handle this value with multiple clipboard values. The way it's handled in this PR as a result is probably incorrect and will need some changes.

Changes made:

  • osu.Framework/Platform/Clipboard.cs
    • Allow writing custom formats
    • Allow writing multiple formats at the same time
  • osu.Framework/Platform/HeadlessClipboard.cs
    • Added in-memory storage for custom formats
  • osu.Framework/Platform/Windows/WindowsClipboard.cs
    • Write custom formats to win32 clipboard api
    • When copying, will generate clipboard entries for custom formats to support web custom formats
    • When pasting, clipboard will fallback to web custom formats clipboard entries and will try to retrieve the data from those instead (to allow copy paste from browser -> osu framework)
  • osu.Framework/Platform/MacOS/MacOSClipboard.cs
    • Added in-memory fallback for custom formats
  • osu.Framework/Platform/SDL2/SDL2Clipboard.cs
    • Added in-memory fallback for custom formats
  • osu.Framework.Android/AndroidClipboard.cs
    • Added in-memory fallback for custom formats
  • osu.Framework/Platform/ClipboardData.cs
    • Used to pass multiple values into the clipboard

Copy link
Collaborator

@bdach bdach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't even know where to begin with this

osu.Framework/Platform/ClipboardData.cs Outdated Show resolved Hide resolved
osu.Framework/Platform/ClipboardData.cs Show resolved Hide resolved
osu.Framework/Platform/ClipboardData.cs Outdated Show resolved Hide resolved
osu.Framework/Platform/Clipboard.cs Outdated Show resolved Hide resolved
osu.Framework/Platform/Clipboard.cs Outdated Show resolved Hide resolved
osu.Framework/Platform/Windows/WindowsClipboard.cs Outdated Show resolved Hide resolved
osu.Framework/Platform/Windows/WindowsClipboard.cs Outdated Show resolved Hide resolved
osu.Framework/Platform/Windows/WindowsClipboard.cs Outdated Show resolved Hide resolved
osu.Framework/Platform/Windows/WindowsClipboard.cs Outdated Show resolved Hide resolved
osu.Framework/Platform/ClipboardData.cs Outdated Show resolved Hide resolved
osu.Framework/Platform/Windows/WindowsClipboard.cs Outdated Show resolved Hide resolved
osu.Framework/Platform/Windows/WindowsClipboard.cs Outdated Show resolved Hide resolved
@bdach
Copy link
Collaborator

bdach commented Mar 29, 2024

I guess I have no major issues with the code itself but I'm not sure how I would test that this does what it says it does if I wanted to. A test scene or something would go a long way.

@minetoblend
Copy link
Author

minetoblend commented Mar 29, 2024

I guess I have no major issues with the code itself but I'm not sure how I would test that this does what it says it does if I wanted to. A test scene or something would go a long way.

Sure thing, I have a couple questions for testing this

  • For testing the web custom formats, should I just check that the generated clipboard entries match an expected output, or should the test verify that the input is indeed parseable by a web browser?
  • If I actually verify that the clipboard works with browsers, would spinning up a headless browser (i.e. puppeteer with https://github.com/hardkoded/puppeteer-sharp) during the test be fine?
  • Since that particular test would not make much sense on other platforms, can I somehow limit the test to run on windows only?

@bdach
Copy link
Collaborator

bdach commented Mar 29, 2024

For testing the web custom formats, should I just check that the generated clipboard entries match an expected output, or should the test verify that the input is indeed parseable by a web browser?

The test can be semi-manual. I'd just like instructions how to perform it.

If I actually verify that the clipboard works with browsers, would spinning up a headless browser (i.e. puppeteer with https://github.com/hardkoded/puppeteer-sharp) during the test be fine?

Don't do that.

Since that particular test would not make much sense on other platforms, can I somehow limit the test to run on windows only?

There is.

@minetoblend
Copy link
Author

minetoblend commented Mar 29, 2024

The test can be semi-manual. I'd just like instructions how to perform it.

I created an html file which can be used to test this. Note that the clipboard API can only be used from a secure context, so the page needs to be served via https. I hosted it over at https://clipboard-test.tiiny.site/ for convenience

I also added some test cases in ppy/osu/pull/27707 that should hopefully be helpful for this. Additionally, I used InsideClipboard to inspect the contents of my clipboard on windows.

If you want to test things manually, you can use these javascript snippets:

  • Reading from clipboard
// needs to be triggered by a user-caused event (i.e. click)
const [item] = await navigator.clipboard.read();

if(item && item.types.contains('mime type I need')) {
  const blob = await item.getType('mime type I need')
  const text = await blob.text()
}
  • Copy co clipboard
// needs to be triggered by a user-caused event (i.e. click)
const item = new ClipboardItem({
  'text/plain': new Blob(['text content'], { type: 'text/plain' }),
  'web custom/format': new Blob(['custom format content'], { type: 'web custom/format' })
})
await navigator.clipboard.write([item])

@bdach
Copy link
Collaborator

bdach commented Apr 3, 2024

I also added some test cases in ppy/osu/pull/27707 that should hopefully be helpful for this

I would hope something can be included on the framework side too. It can be as simple as just printing the clipboard contents in all available formats out on the screen as a string, but it should live in framework.

@minetoblend
Copy link
Author

minetoblend commented Apr 3, 2024

It can be as simple as just printing the clipboard contents in all available formats out on the screen as a string

Sure thing. By printing the clipboard contents to the screen, do you mean grabbing them from the system clipboard or just getting the list of entries that the WindowsClipboad wants to create? Also, is a regular Test what I should be making to do this? Not sure since there's no assertions and sounds like it should be ran manually instead of being part of the test suite.

@bdach
Copy link
Collaborator

bdach commented Apr 3, 2024

By printing the clipboard contents to the screen, do you mean grabbing them from the system clipboard or just getting the list of entries that the WindowsClipboad wants to create?

Just add a text flow and put the key-value pairs with the clipboard contents in it, inside a standard visual test scene.

Also, is a regular Test what I should be making to do this? Not sure since there's no assertions and sounds like it should be ran manually instead of being part of the test suite.

Yes it's fine to write a test without assertions for this. There are many like it, especially around keyboard layout handling and the like.

The point of this excursion is so that I can test that the functionality works without having to check out the game or something. The framework is a standalone project in principle and should be treated as such (even if in practice it turns out to be mostly subservient to the game's needs).

@natsukagami
Copy link

linux would require an upgrade to SDL3

Looks like we're getting it with #6234 (already in) and ppy/osu#27815 (soon)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants